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Abstract 

Event-based  distributed  systems  are  programmed  to  operate  in  response  to  events.  An  event  notifica¬ 
tion  service  is  an  application-independent  infrastructure  that  supports  the  construction  of  event-based 
systems.  While  numerous  technologies  have  been  developed  for  supporting  event-based  interactions 
over  local-area  networks,  these  technologies  do  not  scale  well  to  wide-area  networks  such  as  the  Internet. 
Wide-area  networks  pose  new  challenges  that  have  to  be  attacked  with  solutions  that  specifically  address 
issues  of  scalability.  This  paper  presents  Siena,  a  scalable  event  notification  service  that  is  based  on  a 
distributed  architecture  of  event  servers.  We  first  present  a  formally  defined  interface  that  is  based  on 
an  extension  to  the  publish/subscribe  protocol.  We  then  describe  and  compare  several  different  server 
topologies  and  routing  algorithms.  We  conclude  by  briefly  discussing  related  work,  our  experience  with 
an  initial  implementation  of  Siena,  and  a  framework  for  evaluating  the  scalability  of  event  notification 
services  such  as  Siena. 
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1  Introduction 

The  event-based  architectural  style  is  well  estab¬ 
lished  and  widely  used.  Several  classes  of  appli¬ 
cations  adopt  an  event-based  architecture,  includ¬ 
ing  integrated  development  environments,  work- 
flow  and  process  support  systems,  software  deploy¬ 
ment  systems,  graphical  user  interfaces,  network 
management  tools,  and  security  monitors.  In  this 
style,  components  are  programmed  as  reactive  ob¬ 
jects  that  perform  actions  in  response  to  certain 
events.  Such  style  is  particularly  suitable  for  ap¬ 
plications  that  are  reactive  by  nature,  such  as  net¬ 
work  and  security  monitors,  and  also  for  systems 
that  integrate  heterogeneous  components  and  thus 
require  loosely  coupled  interaction. 

The  connectivity  provided  by  wide-area  net¬ 
works  such  as  the  Internet  offers  even  stronger  mo¬ 
tivation  for  using  an  event-based  architecture.  New 
applications  can  be  designed  that  take  advantage 
of  the  vast  number  of  information  sources  avail¬ 
able  on-line.  Examples  are  stock  market  analysis 
tools  and  data  mining  and  indexing  tools.  Also,  in 
the  context  of  a  wide-area  network,  existing  appli¬ 
cations  can  be  integrated  at  a  much  higher  scale; 
for  example,  workflow  systems  can  be  federated  for 
companies  that  have  multiple  distributed  develop¬ 
ment  sites  or  even  across  corporate  boundaries. 

The  common  infrastructure  underlying  event- 
based  systems  is  the  event  service.  An  event  service 
is  a  general-purpose  facility  that  provides  for  obser¬ 
vation  and  notification  of  events  among  distributed 
objects.  Numerous  technologies  that  realize  an 
event  service  have  been  developed  and  effectively 
used  for  quite  a  long  time.  However,  most  of  them 
target  local-area  networks.  Extending  the  support 
of  an  event  service  to  a  wide-area  network  cre¬ 
ates  new  challenges  and  trade-offs.  Not  only  does 
the  number  of  objects  and  events  grow  tremen¬ 
dously,  but  also  many  of  the  assumptions  made 
for  local-area  networks,  such  as,  low  latency,  abun¬ 
dant  bandwidth,  homogeneous  platforms,  contin¬ 
uous  reliable  connectivity,  and  centralized  control, 
are  no  longer  valid. 

Some  technologies  address  issues  related  to 
wide-area  services.  Among  them,  are  new  tech¬ 


nologies  such  as  Tibco  [8]  that  specifically  pro¬ 
vide  an  event  service,  but  also,  more  mature  tech¬ 
nologies  such  as  the  USENET  news  infrastructure, 
IP  multicasting,  the  Domain  Name  Service  (DNS), 
that,  although  not  explicitly  targeted  at  this  prob¬ 
lem  domain,  represent  potential  or  partial  solu¬ 
tions.  The  main  problem  with  all  of  these  technolo¬ 
gies  is  that  they  are  either  specific  to  some  appli¬ 
cation  domain  or  not  flexible  enough  to  be  usable 
as  a  generic  infrastructure  for  event-based  applica¬ 
tions. 

This  paper  presents  Siena,  a  project  directed  to¬ 
wards  the  design  and  implementation  of  a  scalable 
general-purpose  event  service.  The  contributions 
of  this  work  are  a  formal  definition  of  an  event  ser¬ 
vice  that  combines  expressiveness  with  scalability 
together  with  the  design  and  implementation  of  the 
architectures  and  algorithms  that  realize  this  event 
service  as  a  distributed  infrastructure.  One  obvi¬ 
ous  issue  that  we  must  face  in  this  research  is  the 
evaluation  of  the  solutions  that  we  propose.  To  this 
end,  we  have  performed  systematic  simulations  of 
our  architectures  and  algorithms  in  several  network 
scenarios. 

The  following  section  gives  the  basics  of  the  Siena 
event  service.  The  paper  then  continues  in  Sec¬ 
tion  3  with  a  formal  definition  of  the  interface 
and  the  semantics  of  the  event  service.  The  ar¬ 
chitectures  and  algorithms  that  realize  the  service 
are  presented  in  Section  4.  Section  5  provides  an 
overview  of  some  related  systems  and  research  top¬ 
ics.  Our  evaluation  effort  and  our  experience  with 
a  prototype  is  presented  in  Section  6.  We  then  con¬ 
clude  in  Section  7  with  some  directions  for  future 
work  and  additional  analysis  and  evaluation. 

2  Event  Service 

An  event  service  is  a  dispatcher  of  event  notifica¬ 
tions.  Applications  that  use  the  event  service  can  be 
interested  parties,  i.e.,  event  consumers,  or  objects 
of  interest,  i.e.,  event  generators,  or  both.  The  dis¬ 
patching  is  regulated  by  advertisements,  subscrip¬ 
tions,  and  publications. 

Figure  1  shows  the  high-level  architecture  of  an 
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object  of  interest  interested  party 


Figure  1:  Event  service 

event  service.  Informally,  objects  of  interest  specify 
the  events  they  intend  to  publish  by  means  of  ad¬ 
vertisements  (1),  while  interested  parties  specify  the 
events  they  are  interested  in  by  means  of  subscrip¬ 
tions  (2).  Objects  of  interest  can  then  publish  noti¬ 
fications  (3),  and  the  event  service  will  take  care  of 
delivering  the  notifications  to  the  interested  parties 
that  subscribed  for  them  (4) .  The  terms  used  in  this 
paper,  in  particular  the  terms  notification,  object  of 
interest,  and  interested  party,  follow  the  framework 
proposed  in  [13]. 

Without  loss  of  generality,  we  will  always  assume 
that  objects  of  interest  are  “active”,  i.e.,  they  au¬ 
tonomously  publish  event  notifications.  Passive 
objects,  such  as  files,  can  participate  in  an  event- 
based  interaction  by  means  of  other  active  objects 
that  act  as  proxies  and  that  notify  events  on  behalf 
of  the  passive  objects.  This  distinction  is  similar  to 
the  one  made  in  IEDI  [2] .  In  any  case,  the  passive 
object  will  not  be  considered  in  the  models. 

2.1  Event  Servers 

The  event  service  can  be  realized  by  connecting 
many  events  servers.  An  application  contacts  the 
event  service  via  one  event  server  also  referred  as 
its  access  point,  (see  Figure  2). 

2.2  Identifiers  and  Handlers 

In  order  for  interested  parties,  objects  of  inter¬ 
est,  and  event  servers  to  communicate,  a  naming 
scheme  must  be  adopted  whereby  objects  can  be 
uniquely  identified.  A  handling  scheme  must  also 
be  adopted  so  that  objects  can  be  contacted  using 


Figure  2:  Internal  architecture  of  the  event  service 


appropriate  communication  protocols. 

The  Siena  event  service  adopts  the  generic 
URI  [1]  form  for  both  its  naming  and  handling 
scheme.  This  means  that  every  object  has  a  URI  as¬ 
sociated  with  it  that  defines  both  the  identity  of  that 
object  and  the  handler  used  by  the  event  service  to 
deliver  a  notification  to  that  object.  For  example,  if 
the  URI  mailto:carzanig@cs.colorado.edu  identifies 
an  object,  then  mailto:carzanig@cs.colorado.edu 
is  both  the  unique  name  of  that  object  and  the 
method  that  the  event  service  uses  to  communicate 
with  that  object.  In  this  case,  in  order  to  send  a  no¬ 
tification  to  that  object,  the  event  service  will  send 
an  e-mail  message  to  carzanig@cs.colorado.edu. 

The  event  service  recognizes  the  most  com¬ 
mon  URI  schemas,  including  mailto  and  http, 
and  thus  implements  the  communication  proto¬ 
cols  implied  by  each  schema.  The  implementa¬ 
tion  of  the  event  service  defines  and  maintains 
the  URIs  corresponding  to  event  servers,  how¬ 
ever  it  does  not  directly  assign  or  maintain  URIs 
for  interested  parties  or  objects  of  interest.  Such 
URIs  are  provided  and  operated  by  clients  them¬ 
selves.  This  means  that  if  a  client  identifies  it¬ 
self  as  mailto:carzanig@cs.colorado.edu,  then  the 
event  service  will  simply  assume  that  the  mailbox 
carzanig@cs.colorado.edu  exists  and  is  directly  ac¬ 
cessible. 
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3  Interface  and  Semantics  of  the 
Event  Service 

The  Siena  event  service  exports  the  following  main 
functions: 

publish(notification  n) 
subscribe(URI  subscriber,  pattern  p) 
unsubscribe  (URI  subscriber,  pattern  p) 
advertise  (URI  publisher,  filter  f) 
unadvertise(URI  publisher,  filter  f) 

In  the  following  subsections  we  present  the  syn¬ 
tax  and  the  semantics  of  these  functions  by  for¬ 
mally  defining  notifications,  filters,  and  patterns 
and  their  role  in  every  function.  We  then  present  a 
formal  definition  of  the  semantics  of  the  event  ser¬ 
vice  showing  how  it  can  affect  scalability. 

3.1  Notifications,  Filters,  and  Patterns 

An  event  notification  is  a  set  of  attributes  in 
which  each  attribute  is  a  triple:  attribute  = 
{name,  type,  value).  For  example,  the  notification 
displayed  in  Figure  3  represents  a  stock  price  varia¬ 
tion  event. 


string 

event 

= finance/ exchanges/ stock 

time 

date 

=  Mar  4  11 :43:3  7  MST1 998 

string 

exchange 

-NYSE 

string 

symbol 

=  DIS 

float 

prior 

=  105.25 

float 

change 

=  -4 

float 

earn 

=  2.04 

Figure  3:  Example  of  a  notification 


In  an  event  notification,  attributes  are  uniquely 
identified  by  their  name.  Attribute  types  belong 
to  a  pre- defined  set  of  types.  A  fixed  set  of  oper¬ 
ators  is  also  defined.  Types  and  operators  are  an 
integral  part  of  the  event  service  definition.  We  do 
not  a  give  a  precise  definition  for  the  types  and  op¬ 
erators  here,  but  instead  simply  assume  those  de¬ 
fined  in  modern  programming  languages.  If  a  is 
an  attribute  of  a  notification,  a.name,  a.type,  and 


a.value  denotes  its  name,  type,  and  value  respec¬ 
tively. 

3.2  Filters 

An  event  filter,  or  simply  a  filter,  defines  a  class  of 
event  notifications  by  specifying  a  set  of  attribute 
names  and  types  and  some  constraints  on  their  val¬ 
ues. 


string 

event 

*=  finance/exchanges/* 

string 

exchange 

==  NYSE 

string 

symbol 

==  DIS 

float 

change 

<  0 

Figure  4:  Example  of  an  event  filter 

Figure  4  shows  a  filter  that  selects  negative  stock 
price  variations  for  a  specific  stock  on  a  specific  ex¬ 
change.  More  formally,  a  filter  is  made  of  a  set  of  at¬ 
tribute  filters.  Each  attribute  filter  specifies  a  name, 
a  type,  a  boolean  binary  operator,  and  a  value  for 
an  attribute:  attr-filter  =  [name,  type,  operator, 
value).  In  an  event  filter,  there  can  be  more  than 
one  attribute  filter  with  the  same  name.  For  an  at¬ 
tribute  filter  a,  a.match-op{operandx,operand2)  de¬ 
notes  the  application  of  the  operator  defined  by  a 
to  operandi  and  operand2  ■ 

3.3  Patterns 

A  pattern  of  events  is  defined  by  combining  a  set  of 
event  filters  using  filter  combinators. 


string 

event 

*=  finance/ exchanges/* 

string 

symbol 

==  MSFT 

float 

change 

<  0 

and  then 


string 

event 

*=  finance/exchanges/* 

string 

symbol 

==  NSCP 

float 

change 

>  0 

Figure  5:  Example  of  a  pattern  of  events 


An  example  of  a  pattern  that  combines  two  filters 
into  a  sequence  is  shown  in  Figure  5.  More  formally, 
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an  event  filter  is  itself  a  pattern,  and  any  two  pat¬ 
terns  can  be  combined  to  form  another  pattern  by 
means  of  a  combinator.  Intuitively,  while  a  filter  se¬ 
lects  one  event  notification  at  a  time,  a  pattern  can 
select  several  notifications  that  together  match  an 
algebraic  combination  of  filters. 

We  say  that  a  pattern  is  simple  when  it  contains 
only  one  event  filter.  Also,  since  subscriptions  sub¬ 
mit  patterns  to  the  event  service,  we  say  that  a  sub¬ 
scription  is  simple  when  it  requests  a  simple  pat¬ 
tern  or  compound  when  it  requests  a  pattern  with 
two  or  more  filters. 

For  the  purpose  of  this  paper,  we  will  only  discuss 
the  and  then  or  sequence  combinator  that  construct 
patterns  matching  a  temporal  sequence  of  events. 

3.4  Compatibility  Relations 

In  order  to  give  the  precise  semantics  of  the  event 
service,  we  must  introduce  and  define  the  concept 
of  compatibility  between  notifications  and  sub¬ 
scriptions,  and  between  subscriptions  and  adver¬ 
tisements.  The  compatibility  between  notifications 
and  subscriptions  defines  the  semantics  of  sub¬ 
scriptions  and  comes  into  play  because  the  main 
job  of  the  event  service  is  to  decide  whether  or 
not  notifications  that  are  published  match  any  sub¬ 
scription  submitted  by  an  interested  party.  In  case  a 
notification  matches  some  subscriptions,  the  event 
service  routes  the  notification  towards  all  the  inter¬ 
ested  parties  that  posted  such  subscriptions.  The 
compatibility  between  subscriptions  and  adver¬ 
tisements  is  also  important  because,  in  setting  up 
the  routing  information,  the  event  service  takes  ad¬ 
vertisements  into  account  to  see  if  they  are  relevant 
to  any  subscription.  The  compatibility  between 
subscriptions  and  advertisements  subsumes  a  rela¬ 
tion  between  notifications  and  advertisements  that 
defines  the  semantics  of  advertisements. 

The  following  sections  define  what  it  means  for 
a  notification  to  be  compatible  with  a  subscription 
and  for  a  subscription  to  be  compatible  with  an 
advertisement.  Initially  we  consider  only  simple 
subscriptions  (i.e.,  event  filters)  and  then  extend 
the  compatibility  relations  to  compound  subscrip¬ 


tions. 

3.4.1  Notifications  vs.  Subscriptions 

Let  N  be  the  domain  of  notifications  and  S0  the  set 
of  all  the  simple  subscriptions.  We  define  the  fol¬ 
lowing  binary  relation: 

IsCompatibleff  C  J\f  x  So 

For  brevity,  we  represent  the  relation 
I sC ompatibleff  with  the  symbol  ‘c^’.  When  a 
notification  n  is  compatible  with  a  subscription 
s,  we  also  say  that  s  covers  n,  and  we  denote  with 
N(s)  Cj\f  the  set  of  notifications  n  covered  by  s. 

We  define  the  semantics  of  by  defining  N(s) 
as  follows: 

N(s)  =  {n  €  N  :  Vces  €  s  :  3an  €  n  : 

as.name  =  an.name  A  as.type  =  an.type 

A  as  .match-op{an. value,  as. value)}  (1) 

This  mandates  that  all  attributes  in  the  subscrip¬ 
tion  appear  by  name  in  the  notification  and  that 
they  match  by  type  and  value.  The  notification  can 
also  contain  other  attributes  that  are  not  specified 
in  the  subscription. 

3.4.2  Subscriptions  vs.  Advertisements 

We  first  define  the  semantics  of  advertisements 
similarly  to  what  we  have  done  in  the  previous  sec¬ 
tion  for  subscriptions.  Let  ^4  be  the  domain  of  ad¬ 
vertisements  and  a  e  dan  advertisement.  We  de¬ 
fine  the  set  of  notification  defined  (or  covered)  by  a: 

N(a)  =  {n  €  Af  :  Va„  G  n  :  3aa  €  a  : 

an.name  =  aa-name  A  an.type  =  aa-type 
A  aa.match-op(an.value,  aa.value)} 

(2) 

This  says  that  an  advertisement  covers  all  the  no¬ 
tifications  that  have  a  set  of  attributes  included 
(present  by  name  and  matching  by  value)  in  the  set 
of  attributes  of  the  advertisement. 
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Given  the  definition  of  N  (a)  we  can  easily  define 
IsCompatibleg  (ctj  for  short),  the  compatibility  re¬ 
lation  between  subscription  and  advertisements: 

CjC  So  X  A 

Intuitively,  the  compatibility  between  a  subscrip¬ 
tion  s  and  an  advertisement  a  corresponds  to  the 
relation  between  the  two  sets  of  notifications  de¬ 
fined  by  s  and  a  respectively,  thus: 

s  dj  a  N(a)  fl  N(s)  f  0  (3) 

This  says  that  a  subscription  s  is  compatible  with  an 
advertisement  a  whenever  the  set  of  notifications 
defined  by  a,  N(a),  includes  one  or  more  notifica¬ 
tions  that  are  also  covered  by  s.  When  a  subscrip¬ 
tion  s  is  compatible  with  an  advertisement  a,  we 
also  say  that  a  covers  s. 

3.5  Semantics  of  the  Service 

In  this  section  we  discuss  the  behavior  of  the  event 
service  in  response  to  advertisements,  subscrip¬ 
tions,  and  notifications.  We  have  studied  and  im¬ 
plemented  two  alternative  semantics: 

•  subscription-based,  and 

•  advertisement-based. 

These  two  behaviors  define  two  different  event 
services.  The  reason  to  present  both  and  not  to 
make  a  definite  choice  here  is  that  these  two  se¬ 
mantics  impose  different  requirements  upon  the 
implementation  of  the  event  service,  resulting  in 
different  architectures  with  different  degrees  of 
scalability.  At  this  point,  we  do  not  have  enough  ex¬ 
perience  in  using  the  event  service  to  know  which 
one  is  more  suitable,  flexible,  and  scalable.  It  might 
also  make  sense  to  provide  both  of  them  and  let  the 
user  choose  which  one  works  best  for  each  particu¬ 
lar  situation. 

3.5.1  Subscription-based  Event  Service 

In  the  subscription-based  event  service,  only  sub¬ 
scriptions  determine  the  semantics  of  the  service. 


Advertisements  may  be  used  by  the  event  service 
(e.g.,  to  optimize  the  routing  of  subscriptions),  but 
they  are  not  required.  The  event  service  will  guar¬ 
antee  the  delivery  of  a  notification  to  all  interested 
parties  that  have  subscribed  for  it.  Referring  to  the 
compatibility  relation  between  notifications  and 
subscriptions,  the  event  service  will  deliver  a  noti¬ 
fication  n  to  an  interested  party  X  if  and  only  if: 

1.  X  subscribes  for  s;  and 

2.  n  s. 

3.5.2  Advertisement-based  Event  Service 

In  the  advertisement-based  event  service,  both  ad¬ 
vertisements  and  subscriptions  are  used.  In  partic¬ 
ular,  advertisements  are  used  to  make  notifications 
visible  to  all  the  participants  of  the  event  service. 
More  specifically,  the  event  service  will  guarantee 
the  delivery  of  a  notification  n  posted  by  object  Y 
to  interested  party  X  if  and  only  if 

1.  Y  advertises  a; 

2.  X  subscribes  for  s; 

3.  s  Cg  a;  and 

4.  n  s. 

Note  that  if  an  interested  party  X  sends  a  sub¬ 
scription  s'  that  covers  n,  but  Y  has  never  posted 
any  advertisement  a  that  covers  s',  then  the  event 
service  will  not  guarantee  the  delivery  of  n  to  X. 

3.6  Patterns 

So  far  we  have  discussed  the  semantics  of  the 
event  service  for  simple  subscriptions,  i.e.,  for  sub¬ 
scriptions  that  are  composed  of  one  event  fil¬ 
ter.  However,  both  the  subscription-based  and  the 
advertisement-based  semantics  can  be  easily  ex¬ 
tended  to  incorporate  patterns. 

As  described  above,  patterns  are  defined  by  pat¬ 
tern  filters,  which  are  expressions  whose  elemen¬ 
tary  terms  are  simple  filters.  Thus,  a  subscrip¬ 
tion  to  a  pattern  filter  can  be  logically  viewed  as 
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a  set  of  separate  subscriptions  to  all  the  elemen¬ 
tary  components  of  that  pattern  filter  plus  a  moni¬ 
tor  that  assembles  sequences  of  notifications,  each 
one  matching  one  of  the  elementary  components 
according  to  the  semantics  of  the  combinators. 
Thus,  the  event  service  will  guarantee  the  delivery 
of  a  pattern  of  notifications  matching  an  event  fil¬ 
ter  only  if  it  can  guarantee  the  delivery  of  all  the 
elementary  components  of  the  filter.  Note  that, 
from  this  definition  of  the  semantics  of  patterns, 
the  delivered  pattern  of  notifications  contains  the 
first  notification  matching  each  elementary  com¬ 
ponent. 


3.7  Comments  on  the  Semantics  of  the 
Event  Service 

The  rationale  behind  the  two  semantics  and  their 
extensions  to  patterns  is  to  define  an  event  notifi¬ 
cation  service  that  (1)  behaves  in  an  intuitive  and 
useful  way,  and  (2)  allows  for  an  efficient  and  scal¬ 
able  realization.  In  this  paper,  we  do  not  explore 
the  domain  of  applications  that  would  make  use 
of  an  event  service,  so  we  rely  on  our  previous  re¬ 
search  and  experience  to  justify  the  first  item.  In¬ 
stead,  we  will  elaborate  more  on  the  second  item 
by  showing  how  the  information  provided  by  adver¬ 
tisements  and  subscriptions  with  the  given  seman¬ 
tics  can  be  effectively  used  to  direct  the  communi¬ 
cation  between  event  servers  in  an  efficient  way. 

Timing  and  quality  of  service  are  important,  but 
they’re  not  covered  in  details  in  this  paper.  Timing 
issues  might  arise  when  considering  unsubscrip¬ 
tions  and  unadvertisements.  For  example,  an  in¬ 
terested  party  may  send  an  unsubscription  when 
some  notifications  have  already  been  sent  to  it.  In 
that  case,  the  interested  party  will  probably  receive 
undesired  notifications.  Other  timing  issues  re¬ 
garding  the  ordering  of  notifications  and  thus  pat¬ 
tern  recognition  can  arise  depending  on  the  topol¬ 
ogy  and  latency  of  the  network.  For  the  time  be¬ 
ing  we  will  assume  that  the  event  service  is  able 
within  a  finite  time  to  shuffle  notifications  so  that 
they  are  sent  (and  received)  in  the  correct  temporal 
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By  quality  of  service  we  refer  to  a  number  of  non¬ 
functional  properties  that  do  not  directly  affect  the 
semantics,  but  that  are  nonetheless  of  fundamental 
importance  for  the  practical  realization  and  usage 
of  the  event  service.  A  number  of  other  interface 
functions  will  be  added  to  deal  with  quality  of  ser¬ 
vice  settings  such  as  authentication  and  security, 
and  transactional  communications. 

3.7.1  Rationale:  Expressiveness  vs.  Scalability 

The  rationale  for  our  formal  definition  of  notifica¬ 
tions,  filters,  patterns,  and  compatibility  relations 
goes  beyond  a  clear  specification  of  the  semantics 
of  the  event  service.  The  realization  of  the  event 
service  by  means  of  distributed  event  servers,  re¬ 
quires  to  disseminate  some  information  concern¬ 
ing  subscriptions  and  advertisements  among  event 
servers  in  order  to  control  the  flow  of  notifica¬ 
tions  towards  interested  parties.  In  the  distribu¬ 
tion  of  this  information,  the  compatibility  relations 
together  with  other  similar  relations  between  fil¬ 
ters  (cf  that  defines  the  compatibility  of  two  sim¬ 
ple  subscriptions  and  cjj  that  works  for  two  adver¬ 
tisements),  play  a  fundamental  role.  In  fact,  sim¬ 
ilarly  to  the  optimization  of  queries  in  a  database, 
using  the  compatibility  relations,  the  event  service 
can  optimize  the  deployment  of  filter-  and  pattern- 
matchers  to  minimize  the  usage  of  communication 
and  computation  resources. 

Thus,  for  the  practical  realization  of  the  event 
service  and  for  its  scalability,  it  is  essential  that 
these  relations  can  be  efficiently  implemented.  The 
relations  that  pose  significant  problems  are  clearly 
the  ones  that  involve  two  filters  (e.g.,  ctj);  in  fact, 
computing  n  s  is  just  a  matter  of  applying  the 
filter  defined  by  s  to  n,  which  involves  computing 
a  conjunction  of  simple  predicates  evaluated  for  a 

^his  assumption  would  require  the  existence  of  a  global 
clock,  an  upper  bound  for  the  network  latency  and  the  network 
diameter,  and  sufficiently  big  communication  buffers.  Note  that 
while  these  latter  requirements  can  pose  serious  engineering 
trade-offs,  the  availability  of  high-resolution  GPS  services  makes 
the  first  assumption  very  reasonable  for  most  practical  applica¬ 
tions. 
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particular  instance  of  their  independent  variables. 
On  the  other  hand,  comparing  two  filters,  to  verify 
s  IZ5  a  is  equivalent  to  verifying  the  implication  be¬ 
tween  two  expressions  of  predicates  for  every  pos¬ 
sible  notification. 

Even  in  our  particular  case  in  which  filter  ex¬ 
pressions  are  conjunctions  of  simple  predicates, 
this  problem  can  be  very  hard  to  solve  depending 
on  the  nature  of  types  and  operators  that  consti¬ 
tute  the  simple  predicates.  Given  an  attribute  bi¬ 
ter  /1  =  (N,T,Op,V)  of  name  N,  type  T,  oper¬ 
ator  Op  and  value  V,  and  another  attribute  biter 
f2  =  (N.  T,  Op',  V1)  having  the  same  name  and  type 
plus  operator  Op1  and  value  V' ,  we  want  to  be  able 
to  decide  whether  or  not  the  brst  biter  implies  the 
second: 

(/1  ^/2)»VieT:  Op(x,  V )  =*  Op'(x,  V ') 

Good  operators  are  those  that  debne  equivalence 
relations  and  order  relations  on  totally  ordered  sets. 
The  usual  set  of  basic  types  found  in  a  modern 
programming  language  (numbers,  strings,  chars, 
booleans,  etc.)  and  the  usual  operators  (equality, 
inequality,  regular  expression  match  for  strings), 
satisfy  this  constraint  and  also  constitute  a  quite  ex¬ 
pressive  vocabulary  for  biters. 

Other  systems  adopt  different  notibcation  mod¬ 
els  and  different  bitering  capabilities.  As  a  con¬ 
sequence,  they  realize  different  degrees  of  expres¬ 
siveness  and  scalability.  Section  5  comments  on 
some  of  these  choices  with  respect  to  the  expres¬ 
siveness/scalability  spectrum. 


4  Topologies  and  Algorithms 

The  Siena  event  service  is  architected  as  a  dis¬ 
tributed  system.  This  section  presents  some  alter¬ 
native  realizations  in  which  many  event  servers  co¬ 
operate  to  provide  a  network-wide  event  service. 


4. 1  Server  Topologies 

4.1.1  Hierarchical  Server  Topology 

A  natural  way  of  connecting  event  servers  is  accord¬ 
ing  to  a  hierarchical  topology;  for  instance,  this  is 
the  topology  of  the  distributed  implementation  of 
the  JEDI  event  dispatcher  [2],  As  shown  in  Figure  6, 
each  server  in  a  hierarchical  topology  has  a  number 
clients  that  can  be  either  normal  objects  of  interest 
or  interested  parties  or  other  event  servers.  In  addi¬ 
tion  to  these  connections,  a  server  could  also  have  a 
special  connection  to  a  parent  server  (the  only  out¬ 
going  arrow). 


Figure  6:  Hierarchical  server  topology 

It  is  important  to  note  that  in  this  topology,  a 
server  does  not  distinguish  between  other  servers 
and  its  clients,  and  thus  it  treats  those  servers  as 
clients.  Practically,  this  means  that  a  ‘parent’  server 
will  be  able  to  receive  notibcations,  subscriptions, 
and  advertisements  from  all  its  clients,  but  it  will 
send  only  notibcations  to  its  clients. 


4. 1 .2  Acyclic  Peer-to-Peer  Server  Topology 

In  the  acyclic  peer-to-peer  topology,  servers  com¬ 
municate  with  each  other  as  peers,  thus  allowing 
a  bi-directional  bow  of  subscriptions  and  adver¬ 
tisements  as  well  as  notibcations.  Figure  7  shows 
an  acyclic  peer-to-peer  topology  of  servers.  Once 
again,  notice  the  different  kinds  of  communication 
occurring  between  clients  and  servers  and  among 
servers. 
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Figure  7:  Acyclic  peer-to-peer  server  topology 

4. 1 .3  Generic  Peer-to-Peer  Server  Topology 

The  generic  peer-to-peer  topology  allows  the  same 
type  of  server-to-server  communication  intro¬ 
duced  by  the  acyclic  peer-to-peer,  but  in  addition 
to  that,  it  allows  any  pattern  of  connections  be¬ 
tween  servers  (see  Figure  8). 


Figure  8:  Generic  peer-to-peer  server  topology 


4.2  Routing  Techniques 

Once  connected,  server  must  exchange  notifica¬ 
tions,  subscriptions,  and  advertisements  to  realize 
the  service.  This  section  presents  the  algorithms 
that  disseminate  the  proper  information  through¬ 
out  the  network  of  servers  making  sure  that  noti¬ 
fications  are  correctly  delivered.  Note  that  a  net¬ 
work  of  servers  implementing  the  event  service  is 
logically  equivalent  to  a  network  of  routers  con¬ 
necting  sub-nets  and  realizing  multicast  routing.  In 
fact,  the  algorithms  presented  here  are  very  sim¬ 
ilar  in  principle  to  a  combination  of  the  Inter¬ 
net  Group  Management  Protocol  (IGMP  [6])  and 
a  reverse-path  multicast  routing  algorithm  [5,  4]. 


More  details  on  the  similarities  and  differences 
with  network-level  multicasting  can  be  found  in 
Section  5. 

We  have  defined  two  classes  of  algorithms: 

subscription  forwarding:  This  technique  uses 
subscriptions  to  set  the  paths  for  notifications. 
Every  subscription  is  stored  and  forwarded 
from  the  originating  server  to  all  the  servers 
in  the  network  so  to  form  a  tree  that  con¬ 
nects  the  subscriber  to  all  the  servers  in  the 
network.  When  an  object  publishes  a  noti¬ 
fication  that  matches  that  subscription,  the 
notification  is  routed  towards  the  subscriber 
following  in  reverse  the  path  put  in  place  by 
the  subscription; 

advertisement  forwarding:  This  technique  uses 
advertisements  to  set  the  paths  for  subscrip¬ 
tions,  which  in  turn  set  the  paths  for  noti¬ 
fications.  Every  advertisement  is  forwarded 
throughout  the  network,  thereby  forming  a 
tree  that  reaches  every  server.  When  a  server 
receives  a  subscription,  it  propagates  the  sub¬ 
scription  in  reverse  along  the  path  to  the  adver¬ 
tiser,  thereby  activating  the  path.  Notifications 
are  then  forwarded  only  through  the  activated 
paths. 

There  exists  also  the  degenerate  case  of  broad¬ 
casting  notifications,  which  we  do  not  take  into 
consideration.  Unsubscriptions  and  unadvertise¬ 
ments  are  handled  in  a  similar  way  to  undo  the  ef¬ 
fect  of  the  corresponding  subscription  or  advertise¬ 
ment.  As  suggested  by  their  names,  subscription 
forwarding  and  advertisement  forwarding  imple¬ 
ment  the  subscription-based  and  advertisement- 
based  semantics  respectively.  There  are  two  main 
optimization  strategies  for  saving  communication 
and  computation  resources  that  can  be  pursued  by 
applying  these  two  algorithms,  they  are: 

1.  applying  filters  and  matching  patterns  up¬ 
stream:  this  means  filtering  notifications  and 
assembling  patterns  of  notifications  as  close  as 
possible  to  publishers  (see  Figure  9); 
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Figure  9:  Applying  filters  and  patterns  upstream 

2.  replicating  notifications  downstream.  this 
means  multicasting  notifications  as  close  as 
possible  to  subscribers  (see  Figure  10). 


Figure  10:  Multicasting  of  notifications  down¬ 
stream 

The  broadcasting  or  flooding  process  that  char¬ 
acterizes  both  subscription  forwarding  and  ad¬ 
vertisement  forwarding  creates  minimal  spanning 
trees  for  each  source.  The  realization  of  this  process 
depends  upon  the  underlying  topology  of  servers. 
The  solution  is  trivial  in  the  case  of  acyclic  topolo¬ 
gies  (i.e.,  hierarchical  and  acyclic  peer-to-peer),  but 
it  requires  additional  data  structures  and  protocols 
for  the  generic  graph  topology  [3]. 

In  propagating  requests,  servers  maintain  tables 
of  subscriptions  and/or  advertisements.  When  an 
event  server  receives  a  new  request,  say  a  subscrip¬ 
tion,  that  is  already  covered  by  a  previously  served 
one,  the  server  simply  adds  the  subscriber  to  the 
local  list  and  no  other  action  is  taken.  If  no  such 
subscripition  is  present  in  the  tables,  the  new  re¬ 
quest  is  added  to  the  table  and  propagated.  This  al¬ 
lows  to  prune  of  entire  subtrees  in  the  propagation. 
For  example,  in  the  scenario  of  Figure  10,  when 


server  4  receives  the  forward  of  a  subscription  for 
X  from  server  5  for  the  first  time,  it  propagates  it 
to  server  3,  and  then  all  the  way  towards  server  1. 
However,  when  server  6  sends  the  same  subscrip¬ 
tion  to  server  4,  server  4  stops  the  flooding. 

Servers  perform  other  types  of  optimizations  too. 
For  example,  in  the  advertisement  forwarding  al¬ 
gorithm,  when  a  server  receives  an  advertisement 
from  one  of  its  neighbor  servers  for  which  there  ex¬ 
ist  matching  subscriptions,  the  server  forwards  all 
these  subscriptions  to  the  advertiser.  In  doing  that, 
the  server  tries  to  merge  the  batch  of  subscriptions 
into  a  smaller  number  of  more  generic  subscrip¬ 
tions.  Again,  this  can  be  done  thanks  to  the  sim¬ 
ple  structure  of  filters  and  the  predictable  nature  of 
predicates  and  types  that  can  be  used  in  filters. 

For  the  recognition  of  patterns,  event  servers  try 
to  assemble  patterns  from  smaller  sub-patterns  or 
single  notifications  that  are  already  “available”.  To 
do  this,  servers  rely  on  their  tables  of  advertised 
patterns.  In  short,  when  a  server  receives  a  sub¬ 
scription  that  requests  a  pattern,  it  looks  up  the 
table  of  advertised  filters  trying  to  break  up  that 
sequence  into  smaller  available  filters  or  patterns. 
If  sub-patterns  that  together  form  the  target  se¬ 
quence  have  been  advertised  by  local  clients  or 
neighbor  servers,  the  server  dispatches  subscrip¬ 
tions  for  every  one  of  these  parts  and  starts  up 
a  monitor  that  recognizes  the  requested  sequence 
from  the  sub-sequences. 

Whenever  possible,  a  server  will  push  the  recog¬ 
nition  of  entire  sub-sequences  towards  the  sources 
of  their  components.  For  example,  in  the  scenario 
of  Figure  9,  server  5  notices  that  the  sequence  XY 
can  be  broken  into  X  and  Y  and  that  both  these 
parts  are  available  from  the  same  server  (4),  thus 
server  5  simply  forwards  the  subscription  for  the 
entire  sequence.  Server  4  in  turns  does  exactly  the 
same  thing  forwarding  the  subscription  to  server  3. 
Server  3  notices  that  X  and  Y  are  advertised  by 
two  different  sources,  thus  it  sends  the  two  separate 
simple  subscriptions  and  start  up  the  monitor. 

Once  again  notice  how  the  compatibility  rela¬ 
tions  are  crucial  in  every  step  of  the  forwarding 
techniques.  Also  notice  that  the  way  that  the 
compatibility  relations  define  the  semantics  of  the 
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event  service  is  motivated  by  the  forwarding  tech¬ 
niques.  In  particular,  the  constraints  posed  by  the 
advertisement-based  semantics  make  it  possible 
for  event  servers  to  maintain  advertisements  tables 
that  are  necessary  for  decomposing  and  optimizing 
pattern  recognition. 

5  Related  Work 

In  this  section  we  provide  a  brief  survey  of  tech¬ 
nologies  that  we  believe  are  tightly  related  to  the 
problem  of  wide-area  event  notification,  either  be¬ 
cause  they  attack  the  same  problem  or  because 
they  provide  important  pieces  of  solutions. 

5. 1  Internet  Basic  Technology 

A  number  of  Internet  technologies  are  worth  men¬ 
tioning  because,  if  nothing  else,  they  indeed  realize 
services  on  a  wide-area  network.  Thus,  evenifnone 
of  them  is  geared  towards  an  event  notification  ser¬ 
vice,  it  might  be  worthwhile  to  borrow  their  ideas 
vis-a-vis  scalability. 

5.1.1  Domain  Name  Service 

DNS  is  a  scalable  network  service  that  is  realized 
in  a  distributed  manner.  The  valuable  idea  that 
we  can  borrow  from  DNS  is  its  hierarchical  archi¬ 
tecture.  The  reason  why  the  hierarchical  architec¬ 
ture  works  so  well  for  DNS  is  that  it  maps  very  well 
onto  the  data  that  it  manages.  In  fact,  the  space 
of  domain  names  and  the  space  of  IP  addresses  are 
hierarchical  themselves  and  the  mapping  between 
them  preserves  a  lot  of  the  hierarchical  properties. 
Unfortunately,  the  space  of  notifications  doesn’t  ex¬ 
hibit  any  hierarchical  structure  and,  even  if  we  de¬ 
cided  to  force  this  type  of  structure  (e.g.,  by  defining 
a  mandatory  well  known  attribute  and  a  hierarchi¬ 
cal  set  of  values  for  it),  this  would  not  naturally  map 
onto  a  hierarchical  location  of  objects.  Another  dif¬ 
ferentiator  is  the  essential  read-only  nature  of  the 
DNS  service,  which  does  not  apply  much  to  event 
notification  services. 


5.1.2  USENET  News 

The  USENET  News  system  with  its  main  protocol 
NNTP  [9]  is  perhaps  the  best  example  of  a  scal¬ 
able  user-level  one-to-many  communication  facil¬ 
ity.  USENET  News  messages  are  modeled  after  e- 
mail  messages,  yet  they  provide  additional  infor¬ 
mation  (headers)  that  can  be  used  by  NNTP  com¬ 
mands  to  direct  their  distribution.  NNTP  provides 
both  client-server  and  server-server  commands. 
The  topology  of  news  servers  is  very  similar  to  (and 
in  fact  inspired)  the  acyclic  peer-to-peer  topology. 

The  main  problem  with  USENET  News  and 
NNTP  that  limits  usability  as  an  event  service  is 
that  the  selection  mechanisms  are  not  very  sophis¬ 
ticated.  At  the  protocol  level,  messages  are  filtered 
based  only  on  their  newsgroups  and  on  their  date. 
The  newsgroup  name  space  is  organized  in  a  hi¬ 
erarchy,  and  the  protocol  allows  wild-card  expres¬ 
sions  over  group  names.  Although  group  names 
and  sub-names  reflect  the  general  subject  and  con¬ 
tent  of  messages,  the  filter  that  they  realize  is  too 
coarse-grained  for  most  users  and  definitely  inad¬ 
equate  for  a  general-purpose  event  service.  This 
results  in  unnecessary  transfers  of  entire  groups  of 
messages.  The  service  is  thus  scalable  but  still  quite 
heavyweight,  and  the  time  frame  of  news  propaga¬ 
tion  ranges  from  hours  to  days. 

5.1.3  IP  Multicast 

MBone  is  a  network-level  infrastructure  that  real¬ 
izes  an  efficient  one-to-many  communication  ser¬ 
vice.  An  MBone  or  multicast  IP  address  is  a  virtual 
address  that  corresponds  to  a  group  of  hosts.  Hosts 
can  join  or  leave  a  group  at  any  time.  An  IP  packet 
addressed  to  a  host  group  will  be  delivered  to  all  the 
hosts  in  that  group.  IP  multicast  per  se  is  a  con¬ 
nectionless  best- effort  (unreliable)  service.  A  reli¬ 
able  transport  layer  can  be  implemented  on  top  of 
IP  multicast. 

We  consider  the  IP  multicast  infrastructure  and 
its  routing  algorithms  to  be  the  most  important 
technology  related  to  the  Siena  event  service.  IP 
multicast  may  be  used  as  an  underlying  transport 
mechanism  for  notifications  and  the  ideas  devel- 
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oped  for  routing  multicast  packets  can  be  adapted 
to  solve  the  problem  of  forwarding  notifications  in 
an  event  service.  But  the  IP  multicast  infrastructure 
alone  does  not  qualify  as  an  event  service  because 
of  limitations  in  its  addressing.  The  main  problem 
is  mapping  expressions  of  interest  into  IP  group  ad¬ 
dresses  in  a  scalable  way.  Even  assuming  that  a  sep¬ 
arate  service,  perhaps  similar  to  DNS,  is  available 
for  managing  and  resolving  the  mapping,  the  ad¬ 
dressing  scheme  itself  still  poses  major  limitations 
in  combining  filters  and  patterns.  Because  MBone 
never  relates  two  different  IP  groups,  it  would  not 
be  possible  to  exploit  the  compatibility  relations 
between  filters  and  thus  it  would  not  be  possible  to 
assemble  filters  into  patterns.  Notifications  match¬ 
ing  more  than  one  filter  or  participating  in  more 
than  one  pattern  would  map  into  several  multicast 
addresses,  each  one  being  routed  in  parallel  and 
autonomously  by  MBone,  thus  defeating  the  whole 
purpose  of  the  event  service. 


5.2  Event  Notification  Technologies 

Some  technologies  specifically  realize  an  event  no¬ 
tification  service,  and  some  of  them  also  attempt  to 
extend  their  support  to  wide-area  networks.  To  re¬ 
late  these  systems  to  Siena,  we  adopt  the  classifi¬ 
cation  framework  defined  in  [2]  and  concentrate  in 
particular  on  subscription  languages. 


5.2.1  Channel-based  Subscriptions 

The  simplest  subscription  mechanism  is  the  chan¬ 
nel.  Interested  parties  can  listen  to  a  channel  by 
subscribing  to  it.  An  object  of  interest  publishes 
a  notification  by  addressing  it  to  a  specific  chan¬ 
nel;  as  a  consequence,  the  notification  is  delivered 
to  all  the  parties  that  are  listening  to  that  channel. 
Channel-based  event  services  offer  coarse-grained 
filtering  and  no  patterns.  Since  channels  define 
a  partitioned  address  space  for  notifications,  their 
service  is  equivalent  to  a  reliable  multicast  commu¬ 
nication.  The  CORBA  Event  Service  [12]  adopts  a 
channel  based  architecture. 


5.2.2  Subject-based  Subscription 

Some  systems  extend  the  concept  of  channel  to 
subject-based  addressing.  In  this  case,  event  noti¬ 
fications  contain  a  well-known  attribute  (the  sub¬ 
ject)  that  determines  their  address,  while  the  re¬ 
maining  part  of  the  notification  is  opaque  for  the 
event  service.  The  main  difference  with  respect 
to  channels  is  that  here  subscriptions  can  express 
interest  in  many  (potentially  infinitely  many)  sub¬ 
jects/channels  by  specifying  some  form  of  expres¬ 
sions  to  match  a  subject.  Also,  in  this  model, 
two  different  subscriptions  can  define  overlapping 
sets  of  notifications.  TIBCO  Rendezvous  [8]  adopts 
a  subject-based  subscription  mechanism.  In  TIB 
Rendezvous,  the  subject  is  a  list  of  strings  over 
which  it  is  possible  to  specify  filters  based  on  a  lim¬ 
ited  form  of  regular  expressions;  for  example,  the 
filter  economy .  exchange .  * .  MSFT*  will  select  all  the 
notifications  whose  subject  contains  economy  in  its 
first  position  followed  by  exchange  in  second  posi¬ 
tion,  any  string  in  third  position,  and  a  fourth  string 
that  starts  with  MSFT. 


5.2.3  Content-based  Subscription 

By  extending  the  domain  of  filters  to  the  whole 
content  of  notifications  we  obtain  another  class 
of  subscriptions  called  content-based.  Content- 
based  subscriptions  are  conceptually  very  similar 
to  subject-based  ones.  Elowever,  by  making  the 
whole  structured  content  of  notifications  visible  to 
subscriptions,  they  give  more  freedom  in  the  en¬ 
coding  the  data  upon  which  filters  can  be  applied 
and  which  the  event  service  can  use  for  setting  up 
routing  information.  Moreover,  exposing  the  struc¬ 
ture  of  notifications  makes  their  type  system  (if  any 
is  adopted)  visible  too,  thus,  allowing  more  expres¬ 
sive  and  clear  filters.  Examples  of  systems  that  pro¬ 
vide  this  kind  of  subscription  language  are  JEDI  [2] , 
Yeast  [10],  GEM  [11],  Elvin  [14],  and  Siena  itself. 
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6  Experience 

The  evaluation  of  distributed  software  systems  is  a 
difficult  task,  and  the  Siena  event  service  is  no  ex¬ 
ception.  Because  of  its  highly  distributed  nature, 
it  is  impossible  to  implement  an  event  service  and 
deploy  it  on  a  significant  number  of  nodes  just  to 
see  how  it  works.  Not  only  it  would  be  difficult  to 
measure  its  performance,  but  also  the  cost  of  refin¬ 
ing  its  topologies  and  algorithms  would  be  too  high 
at  least  in  the  early  phases  of  its  design. 

So,  in  order  to  obtain  feedback  early  in  the  design 
and  development  of  Siena,  and  to  have  a  quantita¬ 
tive  evaluation  of  its  topologies  and  algorithm,  we 
adopted  an  approach  that  is  common  practice  in 
the  computer  networks  community  for  the  valida¬ 
tion  of  communication  protocols  and  distributed 
systems  in  general.  We  chose  to  perform  system¬ 
atic  simulations  of  different  combinations  of  server 
topologies  and  dispatching  algorithms  in  several 
network  scenarios. 

6.1  Simulation  Framework 

In  simulating  a  network  scenario,  several  models 
must  be  incorporated  into  the  scenario.  These 
models  define  the  network  topology,  the  layout  of 
event  servers,  the  population  of  applications  (i.e., 
the  distribution  of  interested  parties  and  objects  of 
interest),  their  behavior,  etc.  Clearly,  every  model 
makes  certain  simplifying  assumptions.  Also,  ev¬ 
ery  model  is  characterized  by  a  significant  num¬ 
ber  of  parameters,  which  must  be  adjusted  to  re¬ 
flect  reality  as  faithfully  as  possible.  These  are  the 
most  important  models  we  adopted  in  our  simula¬ 
tion  framework  together  with  their  primary  param¬ 
eters: 

•  network  model :  The  network  model  describes 
the  physical  (wide-area)  communication  net¬ 
work  underlying  the  event  service.  The  net¬ 
work  is  characterized  by  a  graph.  Each  node 
in  the  graph  represents  a  host  or  a  cluster  of 
nodes  connected  by  a  fast  local-area  subnet, 
and  every  edge  represents  a  link  with  its  la¬ 
tency  and  bandwidth.  One  way  of  modeling 


networks  is  to  use  a  real  network  as  a  bench¬ 
mark  for  which  these  parameters  can  be  mea¬ 
sured.  The  other  way,  adopted  in  our  frame¬ 
work,  is  to  use  randomly  generated  graphs 
that  are  good  approximations  of  the  real  net¬ 
work  [15]. 

•  server  model :  We  use  a  layout  of  servers  in 
which  every  network  node  hosts  one  server. 
We  also  assume  that  the  connections  between 
servers  match  the  physical  topology  of  the  net¬ 
work.  This  second  principle  assumes  that  sys¬ 
tem  administrators  have  a  view  of  the  topology 
of  the  immediate  neighborhood  of  their  subnet 
and  that  they  can  configure  the  event  servers 
accordingly. 

•  object  distribution-.  This  model  defines  the  dis¬ 
tribution  of  objects  of  interest  and  interested 
parties  among  subnets.  For  now,  we  use  an  ho¬ 
mogeneous  distribution  of  objects.  Thus,  two 
parameters  are  given  for  the  number  per  node 
of  objects  of  interest  and  interested  parties  re¬ 
spectively. 

•  objects  behavior:  Although  we  could  simu¬ 
late  real  applications,  we  model  applications 
as  Poisson  processes  with  respect  to  genera¬ 
tion  and  consumption  of  events.  Thus,  the  pa¬ 
rameters  that  govern  their  behavior  are  the  av¬ 
erage  time  between  two  requests  and  the  ra¬ 
tio  between  the  number  of  requests  issued  per 
request  type.  This  defines,  for  example,  how 
many  notifications  are  published  for  each  ad¬ 
vertisement. 

•  computation  and  communication  model:  Ob¬ 
jects  communicate  by  exchanging  messages. 
These  messages  can  carry  event  service  re¬ 
quests  (notifications,  subscriptions,  etc.)  as 
well  as  control  messages  or  forwarded  mes¬ 
sages  that  follow  from  some  service  request. 
Objects  execute  their  own  algorithm  in  re¬ 
sponse  to  messages  or  to  the  expiration  of  a 
time-out.  Objects  can  send  messages,  set  time¬ 
outs,  create  other  objects  or  destroy  other  ob¬ 
jects. 
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6.2  Running  Simulations 

Once  a  network  scenario  is  defined,  we  run  several 
simulations  and  collect  traces  of  all  the  low-level 
messages  exchanged  between  processes,  hosts, 
and  subnets.  Then  we  analyze  these  traces  by 
grouping  messages  according  to  some  specific  cri¬ 
teria  (e.g.,  per  host,  per  event  service  request,  per 
type  of  request)  and  by  computing  the  message 
count,  minimum  and  maximum  values,  average 
value,  and  standard  deviation  of  metrics  such  as 
network  cost  and  delay.  The  network  cost  is  a 
per- link  parameter  provided  by  the  network  model 
that  accounts  for  the  usage  of  communication  re¬ 
sources  in  sending  a  message  through  a  link.  In 
our  framework,  we  assume  that  the  cost  is  propor¬ 
tional  to  the  inverse  of  the  bandwidth.  By  varying 
some  parameters  of  the  network  scenario,  such  as 
the  number  of  interested  parties  per  node,  we  can 
plot  these  metrics  and  obtain  indications  of  the  be¬ 
havior  of  that  algorithm. 

AVERAGE  COST  per  (client)  service  /  parties  (sites=100,  (topology=ce,hs,as,aa),  objects=10) 
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Figure  11:  Scalability  of  some  topologies  and  algo¬ 
rithms 

Figure  1 1  shows  the  scalability  of  four  combina¬ 
tion  of  server  topology  and  routing  algorithm:  ce 
=  centralized,  hs  =  hierarchical  +  subscription  for¬ 
warding,  as  =  acyclic  +  subscription  forwarding, 
and  aa  =  acyclic  +  advertisement  forwarding.  All  the 
scenarios  were  executed  on  a  network  of  a  hundred 
nodes.  The  metric  plotted  is  the  average  cost  of 


messages  grouped  on  a  per-request  basis;  the  unit 
of  measure  is  not  really  relevant  since  we  just  want 
to  compare  different  curves.  The  independent  vari¬ 
able  is  the  total  number  of  interested  parties. 

The  simulations  that  we  performed  so  far  clearly 
show  that  our  topologies  for  distributed  event 
services  provide  the  scalability  that  can  not  be 
achieved  with  a  centralized  solution.  In  distin¬ 
guishing  the  various  distributed  topologies,  simu¬ 
lations  show  that  the  peer-to-peer  topologies  dis¬ 
tribute  the  traffic  evenly  among  servers,  as  opposed 
to  the  hierarchical  topology  that  tends  to  over-load 
only  a  few  nodes  (at  the  highest  level  of  the  hierar¬ 
chy)  .  The  simulator  has  been  also  a  very  effective 
testing  tool  for  the  development  of  the  routing  al¬ 
gorithms. 

7  CONCLUSION 

This  paper  has  described  our  work  on  Siena,  a  dis¬ 
tributed,  Internet-scale  event  notification  service. 
We  have  described  the  design  of  the  interface  to  the 
service,  the  algorithms  and  topologies  we  have  de¬ 
signed  to  support  event  notification,  and  our  ongo¬ 
ing  simulation  and  evaluation  work. 

The  simulation  framework  that  we  constructed 
has  helped  us  significantly  in  refining  topologies 
and  algorithms.  Also,  the  simulations  confirm  our 
intuitions  about  the  scalability  of  the  topologies 
and  algorithms  that  we  propose.  However,  we  do 
not  consider  our  evaluation  effort  to  be  complete. 
In  fact,  we  plan  on  continuing  our  evaluation  ef¬ 
fort  by  exploring  the  parameter  space  in  several  di¬ 
rections.  In  particular,  we  are  simulating  different 
ranges  of  behavioral  parameters  to  see  which  algo¬ 
rithms  are  most  sensitive  to  different  classes  of  ap¬ 
plications. 

We  have  implemented  a  prototype  of  Siena  that 
realizes  the  acyclic  topology  with  the  subscription 
forwarding  algorithm.  We  used  this  prototype  as 
the  event  service  of  an  agent-based  deployment 
system  called  the  SoftwareDock  [7].  The  current 
version  of  the  prototype  provides  a  reduced  version 
of  the  notification  model  with  only  strings  and  in¬ 
tegers  and  a  few  operators.  Siena  uses  standard  In- 
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ternet  technology,  so  its  data  model  is  transmitted 
in  XML  format,  and  servers  are  able  to  use  straight 
TCP/IP  as  well  as  SMTP  as  a  transport  layer  for  mes¬ 
sages.  We  plan  on  extending  the  prototype  to  im¬ 
plement  the  advertisement  forwarding  algorithm 
with  a  larger  variety  of  types  and  operators  and 
other  transport  layers  including  HTTP.  This  new 
version  of  the  prototype  will  also  allow  us  to  ap¬ 
ply  the  pattern  matching  optimizations  that  we  dis¬ 
cussed. 
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